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We discuss the chromomagnetic instability problem in the 2SC/g2SC phases and explore 
the phase structure of the gluonic phase which is a candidate for the ground state of dense 
neutral quark matter. 

§1. Introduction 

The phase structure of hot and/or dense QCD is one of the most exciting current 
topics in the field of strong interactions. In particular, the properties of cold and 
dense quark matter are of great interest in astrophysics and cosmology; it is now 
widely accepted that, at moderate densities of relevance for the interior of compact 
stars, quark matter is a color superconductor and has a rich phase structure with 
important implications for compact star physics (for a recent review, see Ref. [ID). 

Bulk matter in the interior of compact stars should be color and electrically 
neutral and be in /3-equilibrium. In the two-flavor case, these conditions separate 
the Fermi momenta of up and down quarks and, as a consequence, the ordinary BCS 
state (2SC) is not always energetically favored over other unconventional states. The 
possibilities include crystalline color superconductivitj^^''^ and gapless color super- 
conductivity (g2SC).'^ However, the 2SC/g2SC phases suffer from a chromomagnetic 
instability, indicated by imaginary Meissner masses of some gluons.^' The instability 
related to gluons of color 4-7 occurs when the ratio of the 2SC gap over the chemical 
potential mismatch, A/dfi, decreases below a value \/2. Resolving the chromomag- 
netic instability and clarifying the nature of true ground state of dense quark matter 
are central issues in the study of color superconductivity.^'' '^-'^''^''^''^''^''I^ 

We will describe the results of recent studies of a chromomagnetic instability 
and a gluonic phas^ (gluonic cylindrical phase II) in dense neutral quark matter. 

§2. Model, Formalism and Numerical Results 

In order to study the chromomagnetic instability and phases with gluonic vector 
condensates, we use a gauged Nambu.Jona-Lasinio (NJL) model with massless up 
and down quarks: 

C = i^{ilp+ /i7°)^ + GD{i>il^ee'>Ci^'^){^Ci-i^ee'>i,) - (2-1) 

where the quark field ip carries flavor = 1, . . . Nf with Nf = 2) and color (a, (5 = 
1, . . . Nc with Nc = 3) indices, C is the charge conjugation matrix; (e)*^ = e*^ and 
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^^b^a/B _ ^bafS antisymmetric tensors in flavor and color spaces, respectively. 

The diquark coupling strength in the scalar color-antitriplet channel is denoted by 
Gd- The covariant derivative and the field strength tensor are defined as 

D^ = d^- igA^T^ F^, = d,Al - d.A^ + gr'^A^Al. (2-2) 

In NJL-type models without dynamic gauge fields, one has to introduce color and 
electric chemical potentials by hand to ensure color- and electric-charge neutrality. In 
/3-equilibrated neutral two-flavor quark matter, the elements of the diagonal matrix 
of quark chemical potentials jl are given by 

Hub = fi- Sfi- /is, Hdb = 1^ + Sfl - /is, (2-3) 

with /i = /i — (5/i/3 -|- /ts/3 and 6fi = /ie/2. 

In the mean-field approximation, the effective potential with constant gluonic 
vector condensates is given by 

Or = f2iA,6^,f,a,A'';f,,T) - r?(0, 0, 0, 0, 0), (2-4) 

where 

^ = ^-\f (|^Trln5-Hp) - ^ + '^r^'^r'^A'^AlA'^'^A-. (2-5) 
Here S^^{p) is the inverse full quark propagator in Nambu-Gor'kov space, 

where = diag(l, —1) is a matrix in flavor space. Following the usual convention, 
we have chosen the diquark condensate to point in the third direction in color space. 
In order to evaluate loop diagrams we use a three-momentum cutoff A = 653.3 MeV 
throughout this paper. 

The chromomagnetic instability related to gluons 4-7 and 8 can be viewed as ten- 
dencies toward the vector condensation of (p^ and A^,^'''^-'^^'^ respectively, where 
= y^1/2(j4^ — iA^^,A^ — lAjj^)'^ is the doublet fleld with respect to the residual 
SU(2)c symmetry in the 2SC/g2SC phases. Because of the SU(2)c symmetry, we can 
choose A^ without loss of generality. 

In Fig. [H we plotted the phase diagram of homogeneous neutral two-flavor 
quark matter in the plane of the 2SC gap at dfj. = (Aq) and temperature (T). (The 
parameter Aq is essentially the diquark coupling strength. We neglect color chemical 
potentials throughout, because the present work remains qualitatively unaffected 
by them.) We calculated the squared Meissner masses of gluons from the second 
derivative of the effective potential with respect to {A^) and {A^) and mapped out 
the unstable regions for gluons 6 and 8 on the phase diagram. In the unstable regions, 
it is natural to suggest that the true ground state is given by the global minimum 
of the effective potential, including the gluonic vector condensation. Note that a 
(^^)-condensed phase is called gluonic phase and a (^^)-condensed phase is gauge 
equivalent to the (single plane-wave) LOFF phase.'^''^''^''^ 
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Fig. 1. Left: The phase diagram of a neutral two-flavor color superconductor in the plane of tem- 
perature and At T = 0, the g2SC phase exists in the window 92 MeV < Zio < 134 MeV 
and the 2SC window is given by zio > 134 MeV. The results are plotted for ^ = 400 MeV. 
Middle: The same as the left panel, but the unstable region for gluons 4-7 is depicted by the 
region enclosed by the thick solid line. At T = 0, the g2SC phase and a part of the 2SC phase 
(92 MeV < Zio < 162 MeV) suffer from the chromomagnetic instability. Right: The same as 
the left panel, but the unstable region for the 8th gluon is depicted by the region enclosed by 
the thick solid line. 
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In order to find the gluonic/LOFF 
phases, we solved the gap equations for 
A and {A") [a G (6, 8)] and the neutral- 
ity condition for 5^1 self-consistently and 
computed their free energies.^ (This 
has been done first by Hashimoto and 
Miransky.-*^^') We illustrate the com- 
parison of the free energies of the glu- 
onic/LOFF phases in Fig. [2l The re- 
sult is plotted for = 400 MeV and 
T = as a function of Aq. The gluonic 
phase exists in the window 66 MeV < 
Aq < 162 MeV and is energetically 
more favored than the LOFF phase in 
a wide range of the coupling strength. 
It is also quite interesting to note that 
the gluonic (LOFF) phase could be en- 
ergetically more favored than the NQ 
phase for 66 MeV < Z\o < 92 MeV 
(65 MeV < Z\o < 92 MeV), though the 
chromomagnetic instability does not ex- 
ist in the NQ phase. By computing the 

effective potential as a function of the vector condensates along the self-consistent 
solution of the gap equation for A and the neutrality condition for Sjj,, we revealed 
that it indeed happens in the weak-coupling regime (see Fig. [3]). In the following. 



Fig. 2. The free energies of the 2SC/g2SC 
phase, the LOFF state, and the gluonic 
phase as a function of Zio (measured with 
respect to the unpaired NQ phase). The 
three dots on the bold solid line denote the 
edge of the g2SC window with the normal 
phase, the phase transition point between 
2SC and g2SC phases (Zi — Sjj,), and the 
critical point of the chromomagnetic insta- 
bility (Zi ~ \/^Sfj,). The results are plotted 
for /i = 400 MeV and T = 0. 



*' The upper boundary of the gluonic (LOFF) phase is slightly larger than that of the unstable 
region for gluons 4-7 (8), because the transition from the gluonic (LOFF) phase to the stable 2SC 
phase is of first order. 
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we take account of the gluonic phase only and look at the phase diagram of two- and 
three- flavor quark matter. 



> 




Ao=140MeV \. 



Ao=100MeV \ 



100 200 
B [MeV] 



300 



> 



o -1 
0. 



-2 



Ao=137 MeV- 



Ao=85 MeV 



Ao=100 MeV 



20 40 
q [MeV] 



60 



Fig. 3. The effective potential of the gluonic (LOFF) phase as a function of B = (ff^z) {<i ~ 
{gA^) /{2\/^)). Note that the effective potentials are calculated along the self-consistent solution 
of the gap equation for A and the neutrality consition for Sfi and are measured with respect 
to the 2SC/g2SC/NQ phases at B = (g = 0). The results are plotted for ^ = 400 MeV and 
T = Q. 
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We extended our analysis to nonzero 
temperatures and obtained the schematic 
phase diagram shown in Fig. HI As ex- 
pected, the low-temperature regions of 
the 2SC/g2SC phases are replaced by 
the gluonic phase. We also find that 
the transition between the gluonic phase 
and the 2SC/g2SC phases is of second 
order or weakly first order. Further- 
more, the gluonic phase wins against a 
part of the NQ phase and undergoes a 
strong first-order phase transition into 
the NQ phase. (Here, it should be men- 
tioned that a cutoff artifact appears in 
the gluon sector of the effective poten- 
tial. Therefore it may be impossible 
to distinguish the second-order transi- 
tion from the weak first-order transition. 
However, the cutoff artifact is not signif- 
icantly large, hence we could calculate 
the phase diagram.) 

From the result shown in Fig. U we could imagine that a large region of the 
2SC/g2SC/NQ phases in the intermediately coupled cold dense quark matter are 
replaced by the gluonic phase. To be more explicit and realistic, we used a three- 
flavor gauged NJL model and investigated the 2SC/g2SC/NQ phases in the T-/i 
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4. Schematic phase diagram of neutral 
two-flavor quark matter at = 400 MeV in 
Aq-T plane. In the region enclosed by the 
thick lines, the gluonic phase is energeti- 
cally more favored than the 2SC/g2SC/NQ 
phases. 
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phase diagram (left panel of Fig. [5j). We used a (density-independent) fixed value 
of the strange quark mass {Ms = 500 MeV). The diquark coupling was chosen so 
that the 2SC gap Aq = 80 MeV at T = 0, ^ = 400 MeV and Ms oo. It should 
be noted that only the 2SC/g2SC/NQ phases were included in this analysis. We 
also neglected the dynamical breakdown of chiral symmetry. Nevertheless, at least 
qualitatively, this phase diagram is consistent with those presented in Ref. [TS]) . We 
again studied the gluonic phase in a self-consistent manner and obtained a phase 
diagram displayed in the right panel of Fig. [5j One can see that the unstable g2SC 
phase is replaced by the gluonic phase and, furthermore, the gluonic phase is favored 
over the stable NQ phase in the low-temperature regime. 

50 
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Fig. 5. Left: The phase diagram of neutral three-flavor quark matter in T-^ plane, with the diquark 
coupling chosen so that the 2SC gap zio = 80 MeV at T = 0, = 400 MeV and Ms oo. 
Note that only the 2SC/g2SC phases are included in the analysis. In the region enclosed by the 
thick solid line, the Meissner masses of gluons 4-7 are tachyonic. Right: The same as the left 
panel, but the gluonic phase is taken into account. The transition between the gluonic phase 
and the g2SC phase is of second order or weakly first order. On the other hand, the gluonic 
phase undergoes a strong first-order transition to the NQ phase. 




§3. Summary 

We studied the chromomagnetic instability in the 2SC/g2SC phases at moderate 
density and at finite temperature. We calculated the Meissner masses squared of glu- 
ons 4-7 and 8 and mapped out the unstable regions on the phase diagram. In order 
to resolve the chromomagnetic instability, we investigated the phases with gluonic 
vector condensations, i.e., the gluonic phase and the single plane- wave LOFF phase. 
Using the gauged NJL model and the mean-field approximation, we computed the 
free energies of the gluonic/LOFF phases in a self-consistent manner and explored 
the phase structure: 

• The gluonic phase is favored over the LOFF phase in a wide range of coupling 
strength. 

• The effective potential shows a peculiar behavior as a function of the vector con- 
densations. It is particularly interesting that, in the weak-coupling regime, the glu- 
onic/LOFF phases could be energetically more favored than the chromomagnetically 
stable NQ phase. 

• The strongly first-order transition between the gluonic phase and the NQ phase 
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takes place. On the other hand, the transition from the gluonic phase to the stable 
g2SC is of second order or weakly first-order. 

In addition to the two-flavor case, we also explored the 2SC/g2SC and the gluonic 
phases in neutral three- flavor quark matter. The resulting T-^ phase diagram shows 
clearly that currently known phase diagrams must be significantly altered. 

As we have shown, the gluonic phase is energetically more favored than the 
unstable 2SC/g2SC phases. However, the project of finding the true ground state of 
neutral dense quark matter is still not complete. In order to draw a conclusion, we 
have to investigate LOFF phases with multiple plane waves and other types of the 
gluonic phase.'^''^ It is also indispensable to look at the chromomagnetic stability 
of those phases.!^ 

Although the result obtained from the (gauged) NJL models seems to be qualita- 
tively reasonable, it should be unreliable quantitatively. Therefore, a novel technique 
is required to perform nonperturbative QCD calculations at relevant densities and 
to obtain a plausible QCD phase diagram. 
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